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Introduction 


1.1.  Research  Objectives 

As  part  of  this  research  program  we  proposed  the  development  of  a  network  thermo¬ 
dynamic  stabilization  framework  for  hybrid  control  design  of  large-scale  and  multiagent 
aerospace  systems.  In  particular,  we  concentrated  on  hybrid  control,  hierarchical  control,  im¬ 
pulsive  dynamical  systems,  nonnegative  dynamical  systems,  compartmental  systems,  large- 
scale  systems,  nonlinear  switching  control,  cooperative  control,  and  adaptive  control.  Appli¬ 
cation  areas  include  large  flexible  interconnected  space  structures,  spacecraft  stabilization, 
cooperative  control  of  unmanned  air  vehicles,  network  systems,  swarms  of  air  and  space 
vehicle  formations,  and  pharmacological  systems. 

1.2.  Overview  of  Research 

Controls  research  by  the  Principal  Investigator  [1-45]  has  concentrated  on  network  ther¬ 
modynamics  for  large-scale  and  multiagent  systems.  In  our  research  we  used  and  are  us¬ 
ing  these  results  to  develop  a  coordination  control  framework  for  finite-time  consensus  and 
parallel  formation  for  multiagent  systems  with  switching  information  topologies  involving 
state-dependent  communication  links  for  addressing  communication  link  failures  and  com¬ 
munication  dropouts.  In  addition,  a  new  neuroadaptive  control  architecture  for  nonlinear 
uncertain  dynamical  systems  is  developed.  The  proposed  framework  involves  a  novel  con¬ 
troller  architecture  involving  additional  terms  in  the  update  laws  that  are  constructed  using  a 
moving  time  window  of  the  integrated  system  uncertainty.  These  terms  can  be  used  to  iden¬ 
tify  the  ideal  system  weights  of  the  neural  network  as  well  as  effectively  suppress  and  cancel 
system  uncertainty  without  the  need  for  persistency  of  excitation.  A  nonlinear  parametriza- 
tion  of  the  system  uncertainty  is  considered  and  state  and  output  feedback  neuroadaptive 
controllers  are  developed. 

Semistable  and  finite-time  semistable  protocols  for  dynamical  networks  with  switching 
topologies  are  also  developed.  Specifically,  we  developed  distributed  static  and  dynamic  out¬ 
put  feedback  controller  architectures  for  coordination  control  for  finite-time  consensus,  ren¬ 
dezvous,  and  parallel  formation  for  multiagent  systems  with  switching  information  topologies 
involving  state-dependent  communication  links  for  addressing  communication  link  failures 
and  communication  dropouts.  In  addition,  using  set-valued  supply  rate  maps  and  set-valued 
storage  maps  consisting  of  piecewise  continuous  storage  functions,  dissipativity  properties  for 
discontinuous  dynamical  systems  are  presented.  These  results  are  used  to  develop  feedback 
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interconnection  stability  results  for  discontinuous  systems  by  appropriately  combining  the 
set- valued  storage  maps  for  the  forward  and  feedback  systems.  Finally,  we  developed  a  new 
and  novel  output  feedback  control  framework  for  nonminimum  phase  multivariable  systems 
for  output  stabilization,  command  following,  and  disturbance  rejection. 

Modern  complex  large-scale  dynamical  systems  exist  in  virtually  every  aspect  of  science 
and  engineering,  and  are  associated  with  a  wide  variety  of  physical,  technological,  environ¬ 
mental,  and  social  phenomena,  including  aerospace,  power,  communications,  and  network 
systems,  to  name  just  a  few  examples.  As  part  of  our  research,  we  developed  a  general  stabil¬ 
ity  analysis  and  control  design  framework  for  nonlinear  large-scale  interconnected  dynamical 
systems  using  vector  Lyapunov  function  methods,  vector  dissipativity  theory,  and  decentral¬ 
ized  control  architectures.  Specifically,  we  addressed  continuous-time  and  hybrid  large-scale 
systems  using  finite-time  decentralized  control  architectures,  thermodynamic  modeling,  max¬ 
imum  entropy  control,  and  energy-based  decentralized  control.  In  addition,  we  developed 
a  stability  analysis  and  control  design  framework  for  time-varying  sets  of  nonlinear  time- 
varying  dynamical  systems  using  vector  Lyapunov  functions.  Using  this  framework,  we 
designed  distributed  control  algorithms  for  multivehicle  coordination.  Finally,  we  developed 
dissipativity  notions  for  dynamical  systems  with  discontinuous  vector  fields.  In  particular, 
we  introduce  a  generalized  definition  of  dissipativity  for  discontinuous  dynamical  systems 
with  Lebesgue  measurable  and  locally  essentially  bounded  vector  fields  characterized  by  dif¬ 
ferential  inclusions  involving  Filippov  set-valued  maps  specifying  a  set  of  directions  for  the 
system  velocity  and  admitting  Filippov  solutions  with  absolutely  continuous  curves.  In  fu¬ 
ture  research  this  framework  will  be  used  as  a  design  tool  for  developing  group  coordination 
algorithms  for  multiagent  systems  possessing  a  dynamic  (i.e.,  switching)  topology. 

1.3.  Goals  of  this  Report 

The  main  goal  of  this  report  is  to  summarize  the  progress  achieved  under  the  program 
during  the  past  three  years.  Since  most  of  the  technical  results  appeared  or  will  soon  appear 
in  over  45  archival  journal  and  conference  publications,  we  shall  only  summarize  these  results 
and  remark  on  their  significance  and  interrelationship. 

2.  Description  of  Work  Accomplished 

The  following  partial  research  accomplishments  have  been  completed  over  the  past  three 
years. 
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2.1.  Semistability,  Finite-Time  Stability,  Differential  Inclusions, 
and  Discontinuous  Dynamical  Systems  Having  a  Continuum 
of  Equilibria 

Numerous  engineering  applications  give  rise  to  discontinuous  dynamical  systems.  Specif¬ 
ically,  in  impact  mechanics  the  motion  of  a  dynamical  system  is  subject  to  velocity  jumps 
and  force  discontinuities  leading  to  nonsmooth  dynamical  systems.  In  mechanical  systems 
subject  to  unilateral  constraints  on  system  positions,  discontinuities  occur  naturally  through 
system-environment  interaction.  Alternatively,  control  of  networks  and  control  over  networks 
with  dynamic  topologies  also  give  rise  to  discontinuous  systems  [12,26].  In  particular,  link 
failures  or  creations  in  network  multiagent  systems  result  in  switchings  of  the  communication 
topology.  In  this  case,  the  vector  held  defining  the  dynamical  system  is  a  discontinuous  func¬ 
tion  of  the  state,  and  hence,  system  stability  can  be  analyzed  using  nonsmooth  Lyapunov 
theory  involving  concepts  such  as  weak  and  strong  stability  notions,  differential  inclusions, 
and  generalized  gradients  of  locally  Lipschitz  continuous  functions  and  proximal  subdiffer¬ 
entials  of  lower  semicontinuous  functions. 

In  many  applications  of  discontinuous  dynamical  systems  such  as  mechanical  systems 
having  rigid-body  modes,  isospectral  matrix  dynamical  systems,  and  consensus  protocols  for 
dynamical  networks,  the  system  dynamics  give  rise  to  a  continuum  of  equilibria.  Under  such 
dynamics,  the  limiting  system  state  achieved  is  not  determined  completely  by  the  dynamics, 
but  depends  on  the  initial  system  state  as  well.  For  such  systems  possessing  a  continuum  of 
equilibria,  semistability  [46],  and  not  asymptotic  stability,  is  the  relevant  notion  of  stability. 
Semistability  is  the  property  whereby  every  trajectory  that  starts  in  a  neighborhood  of  a 
Lyapunov  stable  equilibrium  converges  to  a  (possibly  different)  Lyapunov  stable  equilibrium. 

To  address  the  stability  analysis  of  discontinuous  dynamical  systems  having  a  continuum 
of  equilibria,  in  this  research  [9,12,26]  we  extend  the  theory  of  semistability  to  discontinuous 
time-invariant  dynamical  systems.  In  particular,  we  develop  sufficient  conditions  to  guaran¬ 
tee  weak  and  strong  invariance  of  Fillipov  solutions.  Moreover,  we  present  Lyapunov-based 
tests  for  semistability  of  autonomous  differential  inclusions.  In  addition,  we  develop  sufficient 
conditions  for  finite-time  semistability  of  autonomous  discontinuous  dynamical  systems.  Fu¬ 
ture  extensions  will  focus  on  using  these  results  to  develop  a  coordination  control  framework 
for  finite-time  information  consensus  and  parallel  formation  in  dynamical  networks  with 
switching  topologies  involving  state-dependent  communication  links  for  addressing  commu¬ 
nication  link  failures  and  communication  dropouts. 
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2.2.  Semistability  of  Switched  Dynamical  Systems 

An  essential  feature  of  multiagent  network  systems  is  that  these  systems  possess  a  con¬ 
tinuum  of  equilibria  [12].  Since  every  neighborhood  of  a  nonisolated  equilibrium  contains 
another  equilibrium,  a  non-isolated  equilibrium  cannot  be  asymptotically  stable.  Hence, 
asymptotic  stability  is  not  the  appropriate  notion  of  stability  for  systems  having  a  contin¬ 
uum  of  equilibria.  As  discussed  in  Section  2.1,  such  systems  possess  a  continuum  of  equilibria 
and  hence  semistability  [46]  is  the  relevant  notion  of  stability.  It  is  important  to  note  that 
semistability  is  not  equivalent  to  set  stability  of  the  equilibrium  set.  Indeed,  it  is  possible  for 
trajectories  to  approach  the  equilibrium  set  without  any  trajectory  approaching  any  single 
equilibrium  [46] . 

Since  communication  links  among  multiagent  systems  are  often  unreliable  due  to  mul¬ 
tipath  effects  and  exogenous  disturbances,  the  information  exchange  topologies  in  network 
systems  are  often  dynamic.  In  particular,  link  failures  or  creations  in  network  multiagent 
systems  result  in  switchings  of  the  communication  topology.  This  is  the  case,  for  example, 
if  information  between  agents  is  exchanged  by  means  of  line-of-sight  sensors  that  experience 
periodic  communication  dropouts  due  to  agent  motion.  Variation  in  network  topology  intro¬ 
duces  system  discontinuities,  which  in  turn  give  rise  to  switched  dynamical  systems.  In  this 
case,  the  vector  field  defining  the  dynamical  system  is  a  discontinuous  function  of  the  state 
and/or  time,  and  hence,  system  stability  should  involve  analysis  of  semistability  of  switched 
systems  having  a  continuum  of  equilibria. 

In  this  research  [5,6],  we  develop  semistability  and  uniform  semistability  analysis  re¬ 
sults  for  switched  linear  and  nonlinear  systems.  Since  solutions  to  switched  systems  are  a 
function  of  both  the  system  initial  conditions  and  the  admissible  switching  signals,  unifor¬ 
mity  here  refers  to  the  convergence  rate  to  a  Lyapunov  stable  equilibrium  as  the  switching 
signal  ranges  over  a  given  switching  set.  Our  main  results  involve  sufficient  conditions  for 
semistability  and  uniform  semistability  using  multiple  Lyapunov  functions  and  sufficient  reg¬ 
ularity  assumptions  on  the  class  of  switching  signals  considered.  Specifically,  using  multiple 
Lyapunov  functions  whose  derivatives  are  negative  semideffiiite,  semistability  for  switched 
linear  and  nonlinear  systems  are  established.  If,  in  addition,  the  admissible  switching  signals 
have  infinitely  many  disjoint  intervals  of  length  bounded  from  below  and  above,  uniform 
semistability  can  be  concluded. 
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2.3.  A  New  Neuroadaptive  Control  Architecture  for  Nonlinear 
Uncertain  Dynamical  Systems:  Beyond  a-  and  e-Modifications 

Neural  networks  have  been  extensively  used  for  adaptive  system  identification  as  well  as 
adaptive  and  neuroadaptive  control  of  highly  uncertain  systems.  The  goal  of  adaptive  and 
neuroadaptive  control  is  to  achieve  system  performance  without  excessive  reliance  on  system 
models.  The  fundamental  difference  between  adaptive  control  and  neuroadaptive  control 
can  be  traced  back  to  the  modeling  and  treatment  of  the  system  uncertainties  as  well  as 
the  structure  of  the  basis  functions  used  in  constructing  the  regressor  vector.  In  particular, 
adaptive  control  is  based  on  constant ,  linearly  parameterized  system  uncertainty  models  of  a 
known  structure  but  unknown  parameters.  This  uncertainty  characterization  allows  for  the 
system  nonlinearities  to  be  parameterized  by  a  finite  linear  combination  of  basis  functions 
within  a  class  of  function  approximators  such  as  rational  functions,  spline  functions,  radial 
basis  functions,  sigmoidal  functions,  and  wavelets.  However,  this  linear  parametrization  with 
a  given  basis  function  cannot,  in  general,  exactly  capture  the  system  uncertainty. 

To  approximate  a  larger  class  of  nonlinear  system  uncertainty,  the  uncertainty  can  be 
expressed  in  terms  of  a  neural  network  involving  a  parameterized  nonlinearity.  Hence,  in 
contrast  to  adaptive  control,  neuroadaptive  control  is  based  on  the  universal  function  approx¬ 
imation  property,  wherein  any  continuous  nonlinear  system  uncertainty  can  be  approximated 
arbitrarily  closely  on  a  compact  set  using  a  neural  network  with  appropriate  size,  structure, 
and  weights,  all  of  which  are  not  necessarily  known  a  priori.  Hence,  while  neuroadaptive 
control  has  advantages  over  standard  adaptive  control  in  the  ability  to  capture  a  much  larger 
class  of  uncertainties,  further  complexities  arise  when  the  basis  functions  are  not  known.  In 
particular,  the  choice  and  the  structure  of  the  basis  functions  as  well  as  the  size  of  the  neu¬ 
ral  network  and  the  approximation  error  over  a  compact  domain  become  important  issues 
to  address  in  neuroadaptive  control.  This  difference  in  the  modeling  and  treatment  of  the 
system  uncertainties  results  in  the  ability  of  adaptive  controllers  to  guarantee  asymptotic 
closed-loop  system  stability  versus  ultimate  boundness  as  is  the  case  with  neuroadaptive 
controllers. 

To  improve  robustness  and  the  speed  of  adaptation  of  adaptive  and  neuroadaptive  con¬ 
trollers  several  controller  architectures  have  been  proposed  in  the  literature.  These  include 
the  a-  and  e-modification  architectures  used  to  keep  the  system  parameter  estimates  from 
growing  without  bound  in  the  face  of  system  uncertainty.  In  this  research  [10,13,30],  a  new 
neuroadaptive  control  architecture  for  nonlinear  uncertain  dynamical  systems  is  developed. 
Specifically,  the  proposed  framework  involves  a  new  and  novel  controller  architecture  involv¬ 
ing  additional  terms,  or  Q- modification  terms,  in  the  update  laws  that  are  constructed  using 
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a  moving  time  window  of  the  integrated  system  uncertainty.  The  Q-modification  terms  can 
be  used  to  identify  the  ideal  neural  network  system  weights  which  can  be  used  in  the  adaptive 
law.  In  addition,  these  terms  effectively  suppress  system  uncertainty. 

Even  though  the  proposed  approach  is  reminiscent  to  the  composite  adaptive  control 
framework,  the  Q- modification  framework  does  not  involve  filtered  versions  of  the  control 
input  and  system  state  in  the  update  laws  nor  does  it  involve  a  least-squares  exponential 
forgetting  factor.  Rather,  the  update  laws  involve  auxiliary  terms  predicated  on  an  estimate 
of  the  unknown  neural  network  weights  which  in  turn  are  characterized  by  an  auxiliary 
equation  involving  the  integrated  error  dynamics  over  a  moving  time  interval.  Our  results 
address  vector  uncertainty  structures  with  nonlinear  parameterizations.  In  addition,  state 
and  output  feedback  controllers  are  developed.  Finally,  to  illustrate  the  efficacy  of  the 
proposed  approach  we  apply  our  results  to  a  spacecraft  model  involving  an  unknown  moment 
of  inertia  matrix  as  well  as  a  Boeing  unmanned  combat  aerial  vehicle  model  with  an  actuator 
system  failure  and  compare  our  results  with  standard  neuroadaptive  control  methods. 

2.4.  Finite-Time  Semistability,  Filippov  Systems,  and  Consensus 
Protocols  for  Nonlinear  Dynamical  Networks  with  Switching 
Topologies 

Modern  complex  dynamical  systems  are  highly  interconnected  and  mutually  interdepen¬ 
dent,  both  physically  and  through  a  multitude  of  information  and  communication  networks. 
Distributed  decision-making  for  coordination  of  networks  of  dynamic  agents  involving  in¬ 
formation  flow  can  be  naturally  captured  by  graph-theoretic  notions.  These  dynamical 
network  systems  cover  a  very  broad  spectrum  of  applications  including  cooperative  control 
of  unmanned  air  vehicles  (UAV’s),  autonomous  underwater  vehicles  (AUV’s),  distributed 
sensor  networks,  air  and  ground  transportation  systems,  swarms  of  air  and  space  vehicle 
formations,  and  congestion  control  in  communication  networks,  to  cite  but  a  few  examples. 
Hence,  it  is  not  surprising  that  a  considerable  research  effort  has  been  devoted  to  control  of 
networks  and  control  over  networks  in  recent  years. 

Since  communication  links  among  multiagent  systems  are  often  unreliable  due  to  mul¬ 
tipath  effects  and  exogenous  disturbances,  the  information  exchange  topologies  in  network 
systems  are  often  dynamic.  In  particular,  link  failures  or  creations  in  network  multiagent 
systems  result  in  switchings  of  the  communication  topology.  This  is  the  case,  for  example, 
if  information  between  agents  is  exchanged  by  means  of  linc-of-sight  sensors  that  experience 
periodic  communication  dropouts  due  to  agent  motion.  Variation  in  network  topology  is 
introduced  through  control  input  discontinuities,  which  in  turn  give  rise  to  discontinuous 
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dynamical  systems.  In  this  case,  the  vector  held  defining  the  dynamical  system  is  a  discon¬ 
tinuous  function  of  the  state,  and  hence,  system  stability  can  be  analyzed  using  nonsmooth 
Lyapunov  theory  involving  concepts  such  as  weak  and  strong  stability  notions,  differential 
inclusions,  and  generalized  gradients  of  locally  Lipschitz  continuous  functions  and  proximal 
sub  differentials  of  lower  semicontinuous  functions. 

In  many  applications  involving  multiagent  systems,  groups  of  agents  are  required  to 
agree  on  certain  quantities  of  interest.  In  particular,  it  is  important  to  develop  information 
consensus  protocols  for  networks  of  dynamic  agents  wherein  a  unique  feature  of  the  closed- 
loop  dynamics  under  any  control  algorithm  that  achieves  consensus  is  the  existence  of  a 
continuum  of  equilibria  representing  a  state  of  equipartitioning  or  consensus.  Under  such 
dynamics,  the  limiting  consensus  state  achieved  is  not  determined  completely  by  the  dy¬ 
namics,  but  depends  on  the  initial  system  state  as  well.  Information  consensus  protocols  are 
key  in  addressing  rendezvous  problems,  formation  control,  hocking,  and  attitude  alignment 
in  multiagent  systems.  For  such  systems  possessing  a  continuum  of  equilibria,  semistability , 
and  not  asymptotic  stability,  is  the  relevant  notion  of  stability. 

Semistability  is  the  property  whereby  every  trajectory  that  starts  in  a  neighborhood  of  a 
Lyapunov  stable  equilibrium  converges  to  a  (possibly  different)  Lyapunov  stable  equilibrium. 
From  a  practical  viewpoint,  it  is  not  sufficient  to  only  guarantee  that  a  network  converges  to 
a  state  of  consensus  since  steady  state  convergence  is  not  sufficient  to  guarantee  that  small 
perturbations  from  the  limiting  state  will  lead  to  only  small  transient  excursions  from  a 
state  of  consensus.  It  is  also  necessary  to  guarantee  that  the  equilibrium  states  representing 
consensus  are  Lyapunov  stable,  and  consequently,  semistable.  It  is  important  to  note  that 
semistability  is  not  merely  equivalent  to  asymptotic  stability  of  the  set  of  equilibria,  and 
hence,  is  a  more  natural  stability  notion  than  considering  stability  of  the  consensus  subspace. 

To  address  agreement  problems  in  switching  networks  with  state-dependent  topologies, 
in  this  research  [9, 12]  we  extend  the  theory  of  semistability  to  discontinuous  time-invariant 
dynamical  systems  [9].  In  particular,  we  develop  sufficient  conditions  to  guarantee  weak 
and  strong  invariance  of  Fillipov  solutions.  Moreover,  we  present  Lyapunov-based  tests  for 
strong  and  weak  semistability  for  autonomous  differential  inclusions.  In  addition,  we  develop 
sufficient  conditions  for  hnite-time  semistability  of  autonomous  discontinuous  dynamical 
systems  [12].  Achieving  agreement  in  finite  time  allows  the  dynamical  network  to  use  exact 
information  in  addressing  other  system  tasks.  Furthermore,  using  our  consensus  algorithms 
we  develop  a  coordination  framework  for  multivehicle  rendezvous. 
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2.5.  Dissipativity  Theory  for  Discontinuous  Dynamical  Systems: 
Basic  Input,  State,  and  Output  Properties,  and  Finite-Time 
Stability  of  Feedback  Interconnections 

In  control  engineering,  dissipativity  theory  provides  a  fundamental  framework  for  the 
analysis  and  control  design  of  dynamical  systems  using  an  input,  state,  and  output  system 
description  based  on  system-energy-related  considerations.  The  notion  of  energy  here  refers 
to  abstract  energy  notions  for  which  a  physical  system  energy  interpretation  is  not  necessary. 
The  dissipation  hypothesis  on  dynamical  systems  results  in  a  fundamental  constraint  on  their 
dynamic  behavior,  wherein  a  dissipative  dynamical  system  can  deliver  only  a  fraction  of  its 
energy  to  its  surroundings  and  can  store  only  a  fraction  of  the  work  done  to  it. 

Dissipativity  theory  along  with  Lyapunov  stability  theory  for  feedback  interconnections 
of  dissipative  systems  has  been  extensively  developed  for  dynamical  systems  possessing  con¬ 
tinuous  flows  [46].  However,  numerous  engineering  applications  give  rise  to  discontinuous 
dynamical  systems.  Specifically,  in  impact  mechanics  the  motion  of  a  dynamical  system 
is  subject  to  velocity  jumps  and  force  discontinuities  leading  to  nonsmooth  dynamical  sys¬ 
tems  [9].  In  mechanical  systems  subject  to  unilateral  constraints  on  system  positions,  discon¬ 
tinuities  occur  naturally  through  system-environment  interaction.  Alternatively,  open-loop 
and  feedback  controllers  also  give  rise  to  discontinuous  dynamical  systems.  In  particular, 
bang-bang  controllers  discontinuously  switch  between  maximum  and  minimum  control  input 
values  to  generate  minimum-time  system  trajectories,  whereas  sliding  mode  controllers  use 
discontinuous  feedback  control  for  system  stabilization.  In  switched  systems  [5,  6]  switch¬ 
ing  algorithms  are  used  to  select  an  appropriate  plant  (or  controller)  from  a  given  finite 
parameterized  family  of  plants  (or  controllers)  giving  rise  to  discontinuous  systems.  As  for 
dynamical  systems  with  continuous  flows  [46],  dissipativity  theory  can  play  a  fundamental 
role  in  addressing  robustness,  disturbance  rejection,  stability  of  feedback  interconnections, 
and  optimality  for  discontinuous  dynamical  systems. 

In  light  of  the  fact  that  energy  notions  involving  conservation,  dissipation,  and  transport 
also  arise  naturally  for  discontinuous  systems,  it  seems  natural  that  dissipativity  theory  can 
play  a  key  role  in  the  analysis  and  control  design  of  discontinuous  dynamical  systems.  Specif¬ 
ically,  as  in  the  analysis  of  dynamical  systems  with  continuous  flows,  dissipativity  theory  for 
discontinuous  dynamical  systems  can  involve  conditions  on  system  parameters  that  render 
an  input,  state,  and  output  system  dissipative.  In  addition,  robust  stability  for  discontin¬ 
uous  dynamical  systems  can  be  analyzed  by  viewing  a  discontinuous  dynamical  system  as 
an  interconnection  of  discontinuous  dissipative  dynamical  subsystems.  Alternatively,  discon¬ 
tinuous  dissipativity  theory  can  be  used  to  design  discontinuous  feedback  controllers  that 


add  dissipation  and  guarantee  stability  robustness  allowing  discontinuous  stabilization  to  be 
understood  in  physical  terms. 

In  this  research  [8] ,  we  develop  dissipativity  notions  for  discontinuous  dynamical  systems. 
In  particular,  we  introduce  a  generalized  definition  of  dissipativity  for  discontinuous  dynam¬ 
ical  systems  in  terms  of  set-valued  supply  rate  maps  and  set- valued  storage  maps  consisting 
of  locally  Lebesgue  integrable  supply  rates  and  piecewise  continuous  storage  functions,  re¬ 
spectively.  The  collection  of  storage  functions  and  supply  rates  satisfy  a  set  of  dissipation 
inequalities  reflecting  the  fact  that  the  dissipated  generalized  energies  of  a  discontinuous 
dissipative  system  is  nonnegative  and  is  given  by  the  difference  of  what  is  supplied  and  what 
is  stored.  Our  dissipativity  definition  includes  set-valued  connective  supply  maps  consisting 
of  locally  Lebesgue  integrable  connective  supply  rates  to  reflect  the  fact  that  an  inactive 
storage  function,  corresponding  to  a  Filippov  set- valued  map,  can  still  change  since  multiple 
storage  functions  within  a  set-valued  storage  map  have  common  state  variables. 

Next,  we  develop  extended  Kalman- Yakubovich-Popov  set- valued  conditions  in  terms  of 
the  discontinuous  system  dynamics  for  characterizing  passivity  and  nonexpansivity  via  gen¬ 
eralized  Clarke  gradients  of  locally  Lipschitz  continuous  storage  functions  for  discontinuous 
systems.  In  addition,  using  the  concepts  of  dissipativity  for  discontinuous  dynamical  systems 
with  appropriate  set- valued  storage  maps  and  set- valued  supply  rate  maps,  we  construct  set¬ 
valued  Lyapunov  functions  for  discontinuous  feedback  systems  by  appropriately  combining 
the  set-valued  storage  maps  for  the  forward  and  feedback  subsystems.  General  stability 
criteria  are  given  for  Lyapunov,  asymptotic,  and  finite-time  stability  for  feedback  intercon¬ 
nections  of  discontinuous  dynamical  systems.  In  the  case  where  the  set-valued  supply  rate 
map  consists  of  supply  rates  involving  net  system  power  or  weighted  input-output  energy, 
these  results  provide  extensions  of  the  positivity  and  small  gain  theorems  to  discontinuous 
dynamical  systems. 

2.6.  Output  Feedback  Adaptive  Command  Following  for  Nonmin¬ 
imum  Phase  Uncertain  Dynamical  Systems 

Mathematical  models  are  critical  in  capturing  and  studying  physical  phenomena  that 
undergo  spatial  and  temporal  evolution  arising  in  most  applications  of  science  and  engi¬ 
neering.  These  models  are  often  based  on  first-principles  of  physics  and  are  derived  using 
fundamental  physical  laws.  However,  due  to  system  complexity,  nonlinearities,  uncertainty, 
and  disturbances,  first-principle  models  are  often  based  on  simplifying  approximations  re¬ 
sulting  in  system  modeling  errors.  For  systems  where  the  system  model  does  not  adequately 
capture  the  physical  system  due  to  idealized  assumptions,  model  simplification,  and  model 
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parameter  uncertainty,  adaptive  control  methods  can  be  used  to  achieve  system  performance 
without  excessive  reliance  on  system  models. 

Direct  adaptive  controllers  require  less  system  modeling  information  than  robust  con¬ 
trollers  and  can  address  system  uncertainties  and  system  failures.  These  controllers  adapt 
feedback  gains  in  response  to  system  variations  without  requiring  a  parameter  estimation  al¬ 
gorithm.  This  property  distinguishes  them  from  indirect  adaptive  controllers  that  employ  an 
estimation  algorithm  to  estimate  the  unknown  system  parameters  and  adapt  the  controller 
gains.  Direct  adaptive  controllers  can  be  classified  as  either  full  state  feedback  or  output 
feedback  designs. 

Full  state  feedback  designs  assume  knowledge  of  the  state  variables,  and  this  assumption 
leads  to  computationally  simpler  adaptive  controller  algorithms  as  compared  to  output  feed¬ 
back  algorithms.  Output  feedback  direct  adaptive  controllers,  however,  are  required  for  most 
applications  that  involve  high-dimensional  models  such  as  active  noise  suppression,  active 
control  of  flexible  structures,  fluid  flow  control  systems,  and  combustion  control  processes. 
Models  for  these  applications  vary  from  (reasonably)  accurate  low  frequency  models  in  the 
case  of  structural  control  problems,  to  less  accurate  low-order  models  in  the  case  of  active 
control  of  noise,  vibrations,  flows,  and  combustion  processes. 

There  has  been  a  number  of  results  in  recent  decades  focused  on  output  feedback  di¬ 
rect  adaptive  controllers.  These  results  require  an  observer  for  unknown  state  variables,  an 
observer  for  output  tracking  errors,  an  output  predictor,  and/or  estimation  of  Markov  pa¬ 
rameters  that  lead  to  adaptive  control  algorithms  with  varying  sets  of  assumptions.  These 
assumptions  include  knowledge  of  the  relative  degree  of  the  regulated  system  output  and  the 
dimension  of  the  system,  as  well  as  the  requirement  that  the  system  be  minimum  phase  or 
passive. 

Virtually  all  output  feedback  adaptive  controllers  are  developed  under  a  minimum-phase 
assumption.  In  this  research  [32],  we  develop  an  output  feedback  adaptive  control  framework 
for  nonminimum  phase  multivariable  systems  for  output  stabilization,  command  following, 
and  disturbance  rejection.  The  approach  is  based  on  a  nonminimal  state  space  realization 
that  generates  an  expanded  set  of  states  using  the  filtered  inputs  and  filtered  outputs  of 
the  original  system.  Specifically,  a  direct  adaptive  controller  for  the  nonminimal  state  space 
model  is  constructed  using  the  expanded  states  of  the  nonminimal  realization  and  is  shown  to 
be  effective  for  multi-input,  multi-output  nonminimum  phase  systems  with  unstable  dynam¬ 
ics.  The  adaptive  controller  does  not  require  any  model  information  except  for  an  expanded 
compatibility  condition  involving  the  nonminimal  model,  which  is  far  less  restrictive  than 
standard  matching  conditions  for  model  reference  output  feedback  adaptive  control  involving 
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the  actual  system  dynamics.  In  addition,  the  proposed  adaptive  controller  does  not  require 
knowledge  of  the  nonminimum  phase  system  zeros. 

2.7.  A  Neuroadaptive  Control  Architecture  for  Nonlinear  Uncer¬ 
tain  Dynamical  Systems  with  Amplitude,  Rate,  and  Time- 
Delay  Constraints 

Any  electromechanical  control  actuation  device  is  subject  to  amplitude  and/or  rate  con¬ 
straints  leading  to  saturation  nonlinearities  enforcing  limitations  on  control  amplitudes  and 
control  rates.  Actuator  nonlinearities  can  severely  degrade  closed-loop  system  performance, 
and  in  some  cases  drive  the  system  to  instability,  if  not  accounted  for  in  the  control  design 
process.  These  effects  are  even  more  pronounced  for  adaptive  controllers  which  continue  to 
adapt  when  the  feedback  loop  has  been  severed  due  to  the  presence  of  actuator  saturation 
causing  unstable  controller  modes  to  drift,  which  in  turn  leads  to  severe  windup  effects  and 
unacceptable  transients  after  saturation. 

Many  practical  applications  involve  nonlinear  dynamical  systems  with  simultaneous  con¬ 
trol  amplitude  and  rate  saturation.  The  presence  of  control  rate  saturation  may  further 
exacerbate  the  problem  of  control  amplitude  saturation.  For  example,  in  advanced  tacti¬ 
cal  fighter  aircraft  with  high  maneuverability  requirements,  pilot-induced  oscillations  can 
cause  actuator  amplitude  and  rate  saturation  in  the  control  surfaces,  leading  to  catastrophic 
failures. 

In  addition  to  actuator  amplitude  and  rate  saturation,  actuator  time-delay  constraints  as 
well  as  actuator  dynamics  need  to  be  accounted  for  in  the  control  design  process.  Accounting 
for  input  time  delays  is  critical  in  adaptive  control  design  since  they  can  quantify  time 
delay  margin  which  can  translate  in  increasing  system  gain  margins.  Finally,  accounting  for 
actuator  dynamics  can  capture  the  effects  of  slow  actuator  effects  which  can  degrade  system 
performance,  and  potentially  lead  to  instability. 

In  this  research  [36],  we  develop  a  new  model  reference  adaptive  control  architecture  for 
nonlinear  uncertain  dynamical  systems  with  input  actuator  and  time  delay  constraints.  In 
particular,  we  consider  both  linear  and  nonlinear  in  the  parameters  neural  network  approx¬ 
imations  to  design  neuroadaptive  controllers  for  stabilization  and  command  following  in  the 
presence  of  actuator  dynamics  that  can  effectively  account  for  actuator  amplitude  and  rate 
saturation  constraints.  In  addition,  we  extend  our  framework  to  additionally  account  for 
actuator  time-delay  constraints. 
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2.8.  Stability  and  Control  of  Large-Scale  Dynamical  Systems 

Modern  complex  large-scale  dynamical  systems  arise  in  virtually  every  aspect  of  sci¬ 
ence  and  engineering,  and  are  associated  with  a  wide  variety  of  physical,  technological, 
environmental,  and  social  phenomena.  Such  systems  include  large-scale  aerospace  systems, 
power  systems,  communications  systems,  network  systems,  transportation  systems,  large- 
scale  manufacturing  systems,  integrative  biological  systems,  economic  systems,  ecological 
systems,  and  process  control  systems.  These  systems  are  strongly  interconnected  and  consist 
of  interacting  subsystems  exchanging  matter,  energy,  or  information  with  the  environment. 
In  addition,  the  subsystem  interactions  often  exhibit  remarkably  complex  system  behav¬ 
iors.  Complexity  here  refers  to  the  quality  of  a  system  wherein  interacting  subsystems  form 
multiechelon  hierarchical  evolving  structures  exhibiting  emergent  system  properties. 

The  sheer  size,  or  dimensionality,  of  large-scale  dynamical  systems  necessitates  decen¬ 
tralized  analysis  and  control  system  synthesis  methods  for  their  analysis  and  control  design. 
Specifically,  in  analyzing  complex  large-scale  interconnected  dynamical  systems  it  is  often 
desirable  to  treat  the  overall  system  as  a  collection  of  interacting  subsystems.  The  behavior 
and  properties  of  the  aggregate  large-scale  system  can  then  be  deduced  from  the  behaviors 
of  the  individual  subsystems  and  their  interconnections.  Often  the  need  for  such  an  analysis 
framework  arises  from  computational  complexity  and  computer  throughput  constraints.  In 
addition,  for  controller  design  the  physical  size  and  complexity  of  large-scale  systems  im¬ 
poses  severe  constraints  on  the  communication  links  between  system  sensors,  processors, 
and  actuators,  which  can  render  centralized  control  architectures  impractical.  This  leads  to 
consideration  of  decentralized  controller  architectures  involving  multiple  sensor-processor- 
actuator  subcontrollers  without  real-time  intercommunication.  The  design  and  implemen¬ 
tation  of  decentralized  controllers  is  a  nontrivial  task  involving  control-system  architecture 
determination  and  actuator-sensor  assignments  for  a  particular  subsystem,  as  well  as  pro¬ 
cessor  software  design  for  each  subcontroller  of  a  given  architecture. 

In  this  research  [2],  we  develop  a  unified  stability  analysis  and  control  design  framework 
for  nonlinear  large-scale  interconnected  dynamical  systems  based  on  vector  Lyapunov  func¬ 
tion  methods,  vector  dissipativity  theory,  and  decentralized  control  architectures.  The  use 
of  vector  Lyapunov  functions  in  dynamical  system  theory  offers  a  very  flexible  framework 
for  stability  analysis  since  each  component  of  the  vector  Lyapunov  function  can  satisfy  less 
rigid  requirements  as  compared  to  a  single  scalar  Lyapunov  function.  Moreover,  in  the  anal¬ 
ysis  of  large-scale  interconnected  nonlinear  dynamical  systems,  several  Lyapunov  functions 
arise  naturally  from  the  stability  properties  of  each  individual  subsystem.  In  addition,  since 
large-scale  dynamical  systems  have  numerous  input,  state,  and  output  properties  related  to 
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conservation,  dissipation,  and  transport  of  energy,  matter,  or  information,  extending  classical 
dissipativity  theory  to  capture  conservation  and  dissipation  notions  on  the  subsystem  level 
provides  a  natural  energy  flow  model  for  large-scale  dynamical  systems.  Aggregating  the 
dissipativity  properties  of  each  of  the  subsystems  by  appropriate  storage  functions  and  sup¬ 
ply  rates,  allows  us  to  study  the  dissipativity  properties  of  the  composite  large-scale  system 
using  the  newly  developed  notions  of  vector  storage  functions  and  vector  supply  rates. 

Finally,  a  novel  class  of  fixed-order,  energy-based  hybrid  decentralized  controllers  is  pro¬ 
posed  as  a  means  for  achieving  enhanced  energy  dissipation  in  large-scale  vector  lossless 
and  vector  dissipative  dynamical  systems.  These  dynamic  decentralized  controllers  com¬ 
bine  a  logical  switching  architecture  with  continuous  dynamics  to  guarantee  that  the  system 
plant  energy  is  strictly  decreasing  across  switchings.  The  general  framework  leads  to  hybrid 
closed-loop  systems  described  by  impulsive  differential  equations  [2],  In  addition,  we  con¬ 
struct  hybrid  dynamic  controllers  that  guarantee  that  each  subsystem-subcontroller  pair  of 
the  hybrid  closed-loop  system  is  consistent  with  basic  thermodynamic  principles.  Special 
cases  of  energy-based  hybrid  controllers  involving  state-dependent  switching  are  described, 
and  several  illustrative  examples  as  well  as  an  experimental  testbed  is  designed  to  demon¬ 
strate  the  efficacy  of  the  proposed  approach. 

2.9.  Coordination  Control  for  Multiagent  Interconnected  Systems 

Modern  complex  multiagent  dynamical  systems  are  highly  interconnected  and  mutually 
interdependent,  both  physically  and  through  a  multitude  of  information  and  communica¬ 
tion  networks.  Distributed  decision-making  for  coordination  of  networks  of  dynamic  agents 
involving  information  flow  can  be  naturally  captured  by  graph-theoretic  notions.  These  dy¬ 
namical  network  systems  cover  a  very  broad  spectrum  of  applications  including  cooperative 
control  of  unmanned  air  vehicles  (UAV’s),  autonomous  underwater  vehicles  (AUV’s),  dis¬ 
tributed  sensor  networks,  air  and  ground  transportation  systems,  swarms  of  air  and  space 
vehicle  formations,  and  congestion  control  in  communication  networks,  to  cite  but  a  few 
examples.  Hence,  it  is  not  surprising  that  a  considerable  research  effort  has  been  devoted  to 
control  of  networks  and  control  over  networks  in  recent  years. 

A  key  application  area  of  multiagent  network  coordination  within  aerospace  systems 
is  cooperative  control  of  vehicle  formations  using  distributed  and  decentralized  controller 
architectures.  Distributed  control  refers  to  a  control  architecture  wherein  the  control  is 
distributed  via  multiple  computational  units  that  are  interconnected  through  information 
and  communication  networks,  whereas  decentralized  control  refers  to  a  control  architecture 
wherein  local  decisions  are  based  only  on  local  information.  Vehicle  formations  are  typically 
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dynamically  decoupled,  that  is,  the  motion  of  a  given  agent  or  vehicle  does  not  directly  affect 
the  motion  of  the  other  agents  or  vehicles.  The  multiagent  system  is  coupled  via  the  task 
which  the  agents  or  vehicles  are  required  to  perform. 

The  complexity  of  cooperative  manoeuvres  that  multiagent  systems  need  to  perform  as 
well  as  environmental  constraints  often  necessitate  the  design  of  control  algorithms  that  use 
information  on  current  position  and  velocity  of  each  vehicle  to  steer  them  while  maintaining 
a  specified  formation.  In  particular,  for  mobile  agents  operating  in  a  foggy  environment 
or  located  far  from  each  other,  open-loop  visual  control  for  coordinated  motion  becomes 
impractical.  In  this  case,  feedback  control  algorithms  are  required  for  individual  vehicle 
steering  which  determine  how  a  given  vehicle  maneuvers  based  on  positions  and  velocities 
of  nearby  vehicles  and/or  on  those  of  a  formation  leader.  The  leader  could  be  real,  that  is, 
one  of  the  vehicles  in  a  formation  leads  the  others,  or  the  leader  could  be  virtual,  that  is, 
vehicles  synthesize  a  leader  and  the  motions  of  the  vehicles  in  a  formation  are  defined  with 
respect  to  a  virtual  agent  whose  positions  and  velocities  are  known  at  each  instant  of  time. 

Analysis  and  control  design  for  networks  of  mobile  agents  has  received  considerable  at¬ 
tention  in  the  literature.  Common  formations  of  multiagent  systems  include  flocking,  cyclic 
pursuit,  (virtual)  leader  following,  and  rendezvous.  Graph-theoretic  notions  are  essential  in 
the  analysis  and  control  design  for  a  system  of  mobile  agents  performing  a  common  task. 
Several  researchers  have  proposed  different  techniques  for  analyzing  network  systems.  Specif¬ 
ically,  graph  theory  has  been  used  to  model  interconnected  systems  and  analyze  the  stability 
of  formations  of  large  number  of  agents.  In  addition,  potential  functions  have  been  used  to 
analyze  flocking. 

In  this  research  [2],  we  develop  a  stability  analysis  and  control  design  framework  for  multi- 
agent  coordination  predicated  on  vector  Lyapunov  functions.  In  multiagent  systems,  several 
Lyapunov  functions  arise  naturally  where  each  agent  can  be  associated  with  a  generalized 
energy  function  corresponding  to  a  component  of  a  vector  Lyapunov  function.  Furthermore, 
since  a  specified  formation  of  multiple  vehicles  can  be  characterized  by  a  time- varying  set  in 
the  state  space,  the  problem  of  control  design  for  multiagent  coordinated  motion  is  equiva¬ 
lent  to  design  of  stabilizing  controllers  for  time- varying  sets  of  nonlinear  dynamical  systems. 
Thus,  using  a  stability  and  control  design  framework  for  time-varying  sets,  we  design  dis¬ 
tributed  control  algorithms  for  stabilization  of  multi-vehicle  formations.  These  distributed 
control  algorithms  use  only  local  information  of  the  individual  vehicle  relative  position  and 
velocity  with  respect  to  the  leader  in  order  to  maintain  a  specified  formation  for  a  system 
of  multiple  vehicles.  Finally,  we  specialize  the  results  obtained  for  time- varying  sets  to  ad¬ 
dress  stabilization  of  time-invariant  sets  and  develop  stabilizing  control  algorithms  for  static 
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formations  (rendezvous)  of  multiple  vehicles.  The  developed  cooperative  control  algorithms 
are  shown  to  globally  exponentially  stabilize  both  moving  and  static  formations. 

2.10.  Dissipative  Differential  Inclusions,  Set- Valued  Energy  Stor¬ 
age  and  Supply  Rate  Maps,  and  Discontinuous  Dynamical 
Systems 

The  key  foundation  in  developing  dissipativity  theory  for  nonlinear  dynamical  systems 
with  continuous  flows  was  presented  by  Willems  in  his  seminal  two-part  paper  on  dissipative 
dynamical  systems.  In  particular,  Willems  introduced  the  definition  of  dissipativity  for  gen¬ 
eral  nonlinear  dynamical  systems  in  terms  of  a  dissipation  inequality  involving  a  generalized 
system  power  input,  or  supply  rate,  and  a  generalized  energy  function,  or  storage  function. 
The  dissipation  inequality  implies  that  the  increase  in  generalized  system  energy  over  a  given 
time  interval  cannot  exceed  the  generalized  energy  supply  delivered  to  the  system  during  this 
time  interval.  The  set  of  all  possible  system  storage  functions  is  convex  and  every  system 
storage  function  is  bounded  from  below  by  the  available  system  storage  and  bounded  from 
above  by  the  required  energy  supply. 

In  this  research  [23] ,  we  develop  dissipativity  notions  for  dynamical  systems  with  discon¬ 
tinuous  vector  fields.  Specifically,  we  consider  dynamical  systems  with  Lebesgue  measurable 
and  locally  essentially  bounded  vector  fields  characterized  by  differential  inclusions  involving 
Filippov  set- valued  maps  specifying  a  set  of  directions  for  the  system  velocity  and  admitting 
Filippov  solutions  with  absolutely  continuous  curves.  In  particular,  we  introduce  a  gener¬ 
alized  definition  of  dissipativity  for  discontinuous  dynamical  systems  in  terms  of  set-valued 
supply  rate  maps  and  set-valued  storage  maps  consisting  of  locally  Lebesgue  integrablc  sup¬ 
ply  rates  and  piecewise  continuous  storage  functions,  respectively.  The  collection  of  storage 
functions  and  supply  rates  satisfy  a  set  of  dissipation  inequalities  reflecting  the  fact  that  the 
dissipated  generalized  energies  of  a  discontinuous  dissipative  system  is  nonnegative  and  is 
given  by  the  difference  of  what  is  supplied  and  what  is  stored. 

In  addition,  we  introduce  the  notion  of  a  set-valued  available  storage  map  and  a  set-valued 
required  supply  map,  and  show  that  if  these  maps  have  closed  convex  images  they  involve 
single-valued  maps  corresponding  to  the  smallest  available  storage  and  the  largest  required 
supply  of  the  differential  inclusion,  respectively.  Furthermore,  we  show  that  all  system  stor¬ 
age  functions  are  bounded  from  above  by  the  largest  required  supply  and  bounded  from  below 
by  the  smallest  available  storage,  and  hence,  a  dissipative  differential  inclusion  can  deliver  to 
its  surroundings  only  a  fraction  of  its  generalized  stored  energy  and  can  store  only  a  fraction 
of  the  generalized  work  done  to  it.  Finally,  we  develop  analogous  results  for  lossless  differen- 
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tial  inclusions  as  well  as  specialize  our  results  to  switched  differential  inclusions  to  address 
the  notion  of  dissipativity  for  switched  dynamical  systems.  Future  extensions  will  focus  on 
using  discontinuous  dissipativity  theory  as  a  design  tool  for  developing  group  coordination 
algorithms  for  multiagent  systems  possessing  a  dynamic  (i.e.,  switching)  topology. 

2.11.  Heat  Flow,  Work  Energy,  Chemical  Reactions,  and  Ther¬ 
modynamics:  A  Dynamical  Systems  Perspective 

There  is  no  doubt  that  thermodynamics  is  a  theory  of  universal  proportions  whose  laws 
reign  supreme  among  the  laws  of  nature  and  are  capable  of  addressing  some  of  science’s  most 
intriguing  questions  about  the  origins  and  fabric  of  our  universe.  The  laws  of  thermodynamics 
are  among  the  most  firmly  established  laws  of  nature  and  play  a  critical  role  in  the  under¬ 
standing  of  our  expanding  universe.  In  addition,  thermodynamics  forms  the  underpinning 
of  several  fundamental  life  science  and  engineering  disciplines,  including  biological  systems, 
physiological  systems,  chemical  reaction  systems,  ecological  systems,  information  systems, 
and  network  systems,  to  cite  but  a  few  examples.  While  from  its  inception  its  speculations 
about  the  universe  have  been  grandiose,  its  mathematical  foundation  has  been  amazingly 
obscure  and  imprecise  [47].  This  is  largely  clue  to  the  fact  that  classical  thermodynamics  is 
a  physical  theory  concerned  mainly  with  equilibrium  states  and  does  not  possess  equations 
of  motion.  The  absence  of  a  state  space  formalism  in  classical  thermodynamics,  and  physics 
in  general,  is  quite  disturbing  and  in  our  view  largely  responsible  for  the  monomeric  state  of 
classical  thermodynamics. 

In  recent  research  [47],  we  combined  the  two  universalisms  of  thermodynamics  and  dy¬ 
namical  systems  theory  under  a  single  umbrella  to  develop  a  dynamical  system  formalism 
for  classical  thermodynamics  so  as  to  harmonize  it  with  classical  mechanics.  While  it  seems 
impossible  to  reduce  thermodynamics  to  a  mechanistic  world  picture  due  to  microscopic  re¬ 
versibility  and  Poincare  recurrence,  the  system  thermodynamic  formulation  of  [47]  provides 
a  harmonization  of  classical  thermodynamics  with  classical  mechanics.  In  particular,  our  dy¬ 
namical  system  formalism  captures  all  of  the  key  aspects  of  thermodynamics,  including  its 
fundamental  laws,  while  providing  a  mathematically  rigorous  formulation  for  thermodynam¬ 
ical  systems  out  of  equilibrium  by  unifying  the  theory  of  heat  transfer  with  that  of  classical 
thermodynamics.  In  addition,  the  concept  of  entropy  for  a  nonequilibrium  state  of  a  dy¬ 
namical  process  is  defined,  and  its  global  existence  and  uniqueness  is  established.  This  state 
space  formalism  of  thermodynamics  shows  that  the  behavior  of  heat,  as  described  by  the 
conservation  equations  of  thermal  transport  and  as  described  by  classical  thermodynamics, 
can  be  derived  from  the  same  basic  principles  and  is  part  of  the  same  scientific  discipline. 
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Connections  between  irreversibility,  the  second  law  of  thermodynamics,  and  the  entropic 
arrow  of  time  are  also  established  in  [47].  Specifically,  we  show  a  state  irrecoverability  and, 
hence,  a  state  irreversibility  nature  of  thermodynamics.  State  irreversibility  reflects  time- 
reversal  non-invariance,  wherein  time-reversal  is  not  meant  literally;  that  is,  we  consider 
dynamical  systems  whose  trajectory  reversal  is  or  is  not  allowed  and  not  a  reversal  of  time 
itself.  In  addition,  we  show  that  for  every  nonequilibrium  system  state  and  corresponding 
system  trajectory  of  our  thermodynamically  consistent  dynamical  system,  there  does  not 
exist  a  state  such  that  the  corresponding  system  trajectory  completely  recovers  the  initial 
system  state  of  the  dynamical  system  and  at  the  same  time  restores  the  energy  supplied  by 
the  environment  back  to  its  original  condition.  This,  along  with  the  existence  of  a  global 
strictly  increasing  entropy  function  on  every  nontrivial  system  trajectory,  establishes  the 
existence  of  a  completely  ordered  time  set  having  a  topological  structure  involving  a  closed 
set  homeomorphic  to  the  real  line  giving  a  clear  time-reversal  asymmetry  characterization 
of  thermodynamics  and  establishing  an  emergence  of  the  direction  of  time  flow. 

In  this  research  [14],  we  reformulate  and  extend  some  of  the  results  of  [47].  In  par¬ 
ticular,  unlike  the  framework  in  [47]  wherein  we  establish  the  existence  and  uniqueness  of 
a  global  entropy  function  of  a  specific  form  for  our  thermodynamically  consistent  system 
model,  in  this  research  we  assume  the  existence  of  a  continuously  differentiable,  strictly  con¬ 
cave  function  that  leads  to  an  entropy  inequality  that  can  be  identified  with  the  second  law 
of  thermodynamics  as  a  statement  about  entropy  increase.  We  then  turn  our  attention  to 
stability  and  convergence.  Specifically,  using  Lyapunov  stability  theory  and  the  Krasovskii- 
LaSallc  invariance  principle,  we  show  that  for  an  adiabatically  isolated  system  the  proposed 
interconnected  dynamical  system  model  is  Lyapunov  stable  with  convergent  trajectories  to 
equilibrium  states  where  the  temperatures  of  all  subsystems  are  equal.  Finally,  we  present 
a  state-space  dynamical  system  model  for  chemical  thermodynamics.  In  particular,  we  use 
the  law  of  mass-action  to  obtain  the  dynamics  of  chemical  reaction  networks.  Furthermore, 
using  the  notion  of  the  chemical  potential  [14] ,  we  unify  our  state  space  mass-action  kinetics 
model  developed  in  [1,7]  with  our  thermodynamic  dynamical  system  model  involving  en¬ 
ergy  exchange.  In  addition,  we  show  that  entropy  production  during  chemical  reactions  is 
nonnegative  and  the  dynamical  system  states  of  our  chemical  thermodynamic  state  space 
model  converge  to  a  state  of  temperature  equipartition  and  zero  affinity  (i.e.,  the  difference 
between  the  chemical  potential  of  the  reactants  and  the  chemical  potential  of  the  products 
in  a  chemical  reaction). 
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2.12.  A  Variational  Approach  to  the  Fuel  Optimal  Control  Prob¬ 
lem  for  UAV  Formations 

The  pivotal  role  of  unmanned  aerial  vehicles  (UAVs)  in  modern  aircraft  technology  is 
evidenced  by  the  large  number  of  civil  and  military  applications  they  are  employed  in.  For 
example,  UAVs  successfully  serve  as  platforms  carrying  payloads  aimed  at  land  monitoring, 
wildfire  detection  and  management,  law  enforcement,  pollution  monitoring,  and  communi¬ 
cation  broadcast  relay,  to  name  just  a  few. 

A  formation  of  UAVs,  defined  by  a  set  of  vehicles  whose  states  are  coupled  through  a 
common  control  law,  is  often  more  valuable  than  a  single  aircraft  because  it  can  accomplish 
several  tasks  concurrently.  In  particular,  UAV  formations  can  guarantee  higher  flexibility  and 
redundancy,  as  well  as  increased  capability  of  distributed  payloads.  For  example,  an  aircraft 
formation  can  successfully  intercept  a  vehicle  which  is  faster  than  its  chasers.  Alternatively,  a 
UAV  formation  equipped  with  interferometic  synthetic  aperture  radar  (In-SAR)  antennas  can 
pursue  both  along-track  and  cross-track  interferometry,  which  allow  harvesting  information 
that  a  single  radar  cannot  detect  otherwise. 

Path  planning  is  one  of  the  main  problems  when  designing  missions  involving  multiple 
vehicles;  a  UAV  formation  typically  needs  to  accomplish  diverse  tasks  while  meeting  some 
assigned  constraints.  For  example,  a  UAV  formation  may  need  to  intercept  given  targets 
while  its  members  maintain  an  assigned  relative  attitude.  Trajectories  should  also  be  op¬ 
timized  with  respect  to  some  performance  measure  capturing  minimum  time  or  minimum 
fuel  expenditure.  In  particular,  trajectory  optimization  is  critical  for  mini  and  micro  UAVs 
(/xUAVs)  because  they  often  operate  independently  from  remote  human  controllers  for  ex¬ 
tended  periods  of  time  and  also  because  of  limited  amount  of  available  energy  sources. 

In  this  research  [18],  we  provide  a  rigorous  and  sufficiently  broad  formulation  of  the 
optimal  path  planning  problem  for  UAV  formations,  modeled  as  a  system  of  n  6-degrees  of 
freedom  (DoF)  rigid  bodies  subject  to  a  constant  gravitational  acceleration  and  aerodynamic 
forces  and  moments.  Specifically,  system  trajectories  are  optimized  in  terms  of  control  effort, 
that  is,  we  design  a  control  law  that  minimizes  the  forces  and  moments  needed  to  operate 
a  UAV  formation,  while  meeting  all  the  mission  objectives.  Minimizing  the  control  effort  is 
equivalent  to  minimizing  the  formation’s  fuel  consumption  in  the  case  of  vehicles  equipped 
with  conventional  fuel-based  propulsion  systems  and  is  a  suitable  indicator  of  the  energy 
consumption  for  vehicles  powered  by  batteries  or  other  power  sources. 

In  this  research  [18],  we  also  derive  an  optimal  control  law  which  is  independent  of  the 
size  of  the  formation,  the  system  constraints,  and  the  environmental  model  adopted,  and 
hence,  our  framework  applies  to  aircraft,  spacecraft,  autonomous  marine  vehicles,  and  robot 
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formations.  The  direction  and  magnitude  of  the  optimal  control  forces  and  moments  is  a 
function  of  the  dynamics  of  two  vectors,  namely  the  translational  and  rotational  primer 
vectors.  In  general,  finding  the  dynamics  of  these  two  vectors  over  a  given  time  interval  is 
a  demanding  task  that  does  not  allow  for  an  analytical  closed-form  solution,  and  hence,  a 
numerical  approach  is  required.  Our  main  result  involves  necessary  conditions  for  optimality 
of  the  formations’  trajectories. 

2.13.  Output  Feedback  Adaptive  Stabilization  and  Command  Fol¬ 
lowing  for  Minimum  Phase  Uncertain  Dynamical  Systems 

As  discussed  in  Section  2.6,  there  has  been  a  number  of  results  in  recent  decades  focused 
on  output  feedback  direct  adaptive  controllers.  These  results  require  an  observer  for  un¬ 
known  state  variables,  an  observer  for  output  tracking  errors,  an  output  predictor,  and/or 
estimation  of  Markov  parameters  that  lead  to  adaptive  control  algorithms  with  varying  sets 
of  assumptions.  These  assumptions  include  knowledge  of  the  relative  degree  of  the  regulated 
system  output  and  the  dimension  of  the  system,  as  well  as  the  requirement  that  the  sys¬ 
tem  be  minimum  phase  or  passive.  The  main  reason  for  the  minimum  phase  assumption  is 
because  direct  adaptive  controllers  employ  high  gain  feedback  that  can  drive  nonminimum 
phase  systems  to  instability. 

In  this  research  [22],  we  extend  the  disturbance  free  adaptive  control  framework  pre¬ 
sented  in  [37]  to  develop  an  output  feedback  adaptive  control  framework  for  continuous-time 
minimum  phase  multivariable  uncertain  dynamical  systems  with  exogenous  disturbances  for 
output  stabilization  and  command  following.  The  approach  is  based  on  a  nonminimal  state 
space  realization  that  generates  an  expanded  set  of  states  using  the  filtered  inputs  and  fil¬ 
tered  outputs  and  their  derivatives  of  the  original  system.  Specifically,  a  direct  adaptive 
controller  for  the  nonminimal  state  space  model  is  constructed  using  the  expanded  states  of 
the  nonminimal  realization  and  is  shown  to  be  effective  for  multi-input,  multi-output  linear 
dynamical  systems  with  unmatched  disturbances,  unmatched  uncertainties,  and  unstable 
dynamics.  The  proposed  adaptive  control  architecture  requires  knowledge  of  the  open-loop 
system’s  relative  degree  as  well  as  a  bound  on  the  system’s  order.  Several  illustrative  nu¬ 
merical  examples  are  provided  to  demonstrate  the  efficacy  of  the  proposed  approach. 
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2.14.  Robust  Adaptive  Control  Architecture  for  Disturbance  Re¬ 
jection  and  Uncertainty  Suppression  with  Transient  and 
Steady-State  Performance  Guarantees 

One  of  the  fundamental  problems  in  feedback  control  design  is  the  ability  of  the  control 
system  to  guarantee  robust  stability  and  robust  performance  with  respect  to  system  uncer¬ 
tainties  in  the  design  model.  To  this  end,  adaptive  control  along  with  robust  control  theory 
have  been  developed  to  address  the  problem  of  system  uncertainty  in  control- system  design. 
The  fundamental  differences  between  adaptive  control  design  and  robust  control  design  can 
be  traced  to  the  modeling  and  treatment  of  system  uncertainties  as  well  as  the  controller 
architecture  structures. 

In  particular,  adaptive  control  is  based  on  constant  linearly  parameterized  system  un¬ 
certainty  models  of  a  known  structure  but  unknown  variation,  whereas  robust  control  is 
predicated  on  structured  and/or  unstructured  linear  or  nonlinear  (possibly  time- varying) 
operator  uncertainty  models  consisting  of  bounded  variation.  Hence,  for  systems  with  con¬ 
stant  real  parametric  uncertainties  with  large  unknown  variations,  adaptive  control  is  clearly 
appropriate,  whereas  for  systems  with  time-varying  parametric  uncertainties  and  nonpara- 
metric  uncertainties  with  norm  bounded  variations,  robust  control  may  be  more  suitable. 

In  contrast  to  fixed-gain  robust  controllers,  which  are  predicated  on  a  mathematical  model 
of  the  system  uncertainty  and  which  maintain  specified  constants  within  the  feedback  control 
law  to  sustain  robust  stability  and  performance  over  the  range  of  system  uncertainty,  adap¬ 
tive  controllers  directly  or  indirectly  adjust  feedback  gains  to  maintain  closed-loop  stability 
and  improve  performance  in  the  face  of  system  uncertainties.  Specifically,  indirect  adaptive 
controllers  utilize  parameter  update  laws  to  identify  unknown  system  parameters  and  adjust 
feedback  gains  to  account  for  system  variation,  whereas  direct  adaptive  controllers  directly 
adjust  the  controller  gains  in  response  to  plant  variation.  In  either  case,  the  overall  process 
of  parameter  identification  and  controller  adjustment  constitutes  a  nonlinear  control  law  ar¬ 
chitecture,  which  makes  validation  and  verification  of  guaranteed  transient  and  steady-state 
performance,  as  well  as  robustness  margins  of  adaptive  controllers  extremely  challenging. 

While  adaptive  control  has  been  used  in  numerous  applications  to  achieve  system  per¬ 
formance  without  excessive  reliance  on  system  models,  the  necessity  of  high-gain  feedback 
for  achieving  fast  adaptation  can  be  a  serious  limitation  of  adaptive  controllers.  Specifically, 
in  certain  applications  fast  adaptation  is  required  to  achieve  stringent  tracking  performance 
specifications  in  the  face  of  large  system  uncertainties  and  abrupt  changes  in  system  dy¬ 
namics.  This,  for  example,  is  the  case  for  high  performance  aircraft  systems  that  can  be 
subjected  to  system  faults  or  structural  damage  which  can  result  in  major  changes  in  aero- 
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dynamic  system  parameters.  In  such  situations,  high-gain  adaptive  control  is  necessary  in 
order  to  rapidly  reduce  and  maintain  system  tracking  errors.  However,  fast  adaptation  using 
high-gain  feedback  can  result  in  high-frequency  oscillations  which  can  excite  unmodeled  sys¬ 
tem  dynamics  resulting  in  system  instability.  Hence,  there  exists  a  critical  trade-off  between 
system  stability  and  control  adaptation  rate. 

Virtually  all  adaptive  control  methods  developed  in  the  literature  have  averted  the  prob¬ 
lem  of  high-gain  control.  Notable  exceptions  include  the  use  of  a  low-pass  filter  that  effec¬ 
tively  subverts  high  frequency  oscillations  that  can  occur  due  to  fast  adaptation  while  using 
a  predictor  model  to  reconstruct  the  reference  system  model.  In  particular,  this  method  in¬ 
volves  a  robust  adaptive  control  architecture  that  provides  sufficient  conditions  for  stability 
and  performance  in  terms  of  fA-norms  of  the  underlying  system  transfer  functions  despite 
fast  adaptation,  leading  to  uniform  bounds  on  the  £oo-norms  of  the  system  input-output 
signals. 

In  this  research  [19],  a  new  adaptive  control  architecture  for  linear  and  nonlinear  uncer¬ 
tain  dynamical  systems  is  developed  to  address  the  problem  of  high-gain  adaptive  control. 
Specifically,  the  proposed  framework  involves  a  new  and  novel  controller  architecture  involv¬ 
ing  a  modification  term  in  the  update  law  that  minimizes  an  error  criterion  involving  the 
distance  between  the  weighted  regressor  vector  and  the  weighted  system  error  states.  This 
modification  term  allows  for  fast  adaptation  without  hindering  system  robustness.  In  par¬ 
ticular,  we  show  that  the  governing  tracking  closed-loop  system  error  equation  approximates 
a  Hurwitz  linear  time-invariant  dynamical  system  with  input-output  signals.  This  key 
feature  of  our  framework  allows  for  robust  stability  analysis  of  the  proposed  adaptive  control 
law  using  C\  system  theory.  Specifically,  in  the  face  of  fast  adaptation,  uniform  transient  and 
steady-state  system  performance  bounds  are  derived  in  terms  of  C i -norms  of  the  closed-loop 
system  error  dynamics.  We  further  show  that  by  properly  choosing  the  design  parameters  in 
the  modification  term  we  can  adjust  the  bandwidth  of  the  adaptive  controller,  the  transient 
and  steady-state  closed-loop  performance,  and  the  size  of  the  ultimate  bound  of  the  closed- 
loop  system  trajectories  independently  of  the  system  adaptation  rate.  Several  illustrative 
numerical  examples  are  provided  to  demonstrate  the  efficacy  of  the  proposed  approach. 

2.15.  Multistability,  Bifurcations,  and  Biological  Neural  Networks: 
A  Synaptic  Drive  Firing  Model  for  Cerebral  Cortex  Transi¬ 
tion  in  the  Induction  of  General  Anesthesia 

Advances  in  neuroscience  have  been  closely  linked  to  mathematical  modeling  beginning 
with  the  integrate-and-fire  model  of  Lapicque  and  proceeding  through  the  modeling  of  the 
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action  potential  by  Hodgkin  and  Huxley  to  the  current  era  of  mathematical  neuroscience. 
Neuroscience  has  always  had  models  to  interpret  experimental  results  from  a  high-level  com¬ 
plex  systems  perspective;  however,  expressing  these  models  with  dynamic  equations  rather 
than  words  fosters  precision,  completeness,  and  self-consistency.  Nonlinear  dynamical  sys¬ 
tem  theory,  in  particular,  can  provide  a  framework  for  a  rigorous  description  of  the  behavior 
of  large-scale  networks  of  neurons.  A  particularly  interesting  application  of  nonlinear  dy¬ 
namical  systems  theory  to  the  neurosciences  is  to  study  phenomena  of  the  central  nervous 
system  that  exhibit  nearly  discontinuous  transitions  between  macroscopic  states.  One  such 
example  exhibiting  this  phenomenon  is  the  induction  of  general  anesthesia. 

The  rational,  safe,  and  effective  utilization  of  any  drug  in  the  practice  of  medicine  is 
grounded  in  an  understanding  of  the  pharmacodynamics  of  the  drug,  loosely  defined  as  what 
the  drug  does  to  the  body.  A  very  important  measure  of  the  pharmacodynamics  of  any 
drug  is  the  drug  concentration  parameter  EC50,  which  reflects  the  drug  dose  at  which  the 
therapeutic  effect  is  achieved  in  50%  of  the  cases.  This  concept  is  certainly  applicable  for  the 
administration  of  general  inhalational  anesthetics,  where  the  potency  of  the  drug  is  defined 
by  the  minimum  alveolar  concentration  (MAC)  of  the  drug  needed  to  prevent  a  response  to 
noxious  stimuli  in  50%  of  administrations. 

The  MAC  concept  is  intrinsically  embedded  in  a  probabilistic  framework.  It  is  the  con¬ 
centration  at  which  the  probability  of  a  response  to  a  noxious  stimulus  is  0.5.  Typically  the 
MAC  of  a  particular  anesthetic  is  determined  by  administering  various  doses  of  the  agent 
to  a  population  of  patients  and  determining  the  dose  at  which  there  is  a  0.5  chance  of  re¬ 
sponding  to  a  noxious  stimulus.  (Technically,  we  identify  the  concentration  in  the  alveoli, 
the  fundamental  functional  gas  exchange  units  of  the  lung,  at  which  the  chance  of  response 
is  0.5.)  It  has  been  possible,  however,  to  conduct  studies  of  single  subjects,  varying  the 
anesthetic  concentration  and  determining  responsiveness.  When  this  has  been  done,  it  has 
been  noted  that  the  transition  from  responsiveness  to  non-responsiveness  in  the  individual 
patient  is  very  sharp,  almost  an  all-or-none  transition.  This  simply  confirms  the  observa¬ 
tions  of  generations  of  clinicians.  And  this  raises  the  question  of  how  to  account  for  such  a 
transition  in  terms  of  the  known  molecular  properties  of  the  anesthetic  agent. 

Although  general  anesthesia  has  been  used  in  the  clinical  practice  of  medicine  for  over 
150  years,  the  mechanism  of  action  is  still  not  fully  understood  and  is  still  under  considerable 
investigation.  Theories  range  from  a  nonspecific  perturbation  of  the  lipid  bilayer  membrane 
of  neurons,  the  cells  responsible  for  the  “information”  function  of  the  central  nervous  system, 
to  the  interaction  of  the  anesthetic  agent  with  specific  protein  receptors.  Early  theories 
postulated  that  anesthesia  is  produced  by  disturbance  of  the  physical  properties  of  cell 
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membranes.  The  work  of  Meyer  and  Overton  demonstrated  that  for  some  anesthetics  there 
was  a  correlation  between  anesthetic  potency  and  solubility  in  fat-like  solvents.  This  led 
to  a  theory  that  anesthesia  resulted  from  a  nonspecific  perturbation  of  the  lipid  bilayer 
membrane  of  neurons.  Subsequent  research  then  found  that  membrane  proteins  performed 
functions  of  excitability  and  this  led  to  a  focus  on  anesthetic  binding  and  perturbation 
of  hydrophobic  regions  of  membrane  proteins.  Further  research  also  revealed  that  some 
anesthetic  gases  follow  the  Meyer- Overton  correlation  but  do  not  produce  anesthesia  and 
some  Meyer- Overton  gases  are  excitatory  and  can  cause  seizures.  These  results  led  to  the 
more  common  modern  focus  on  the  interaction  of  the  anesthetic  agent  with  specific  protein 
receptors. 

In  particular,  there  has  been  extensive  investigation  of  the  influence  of  anesthetic  agents 
on  the  binding  of  neurotransmitters  to  their  postsynaptic  receptors.  A  plethora  of  recep¬ 
tors  have  been  investigated,  including  receptors  for  glycine,  serotonin  type  2  and  3,  N- 
methyl-d-aspartate  (NMDA),  a-2  adrenoreceptors,  a- amino-3-hydroxy-5- methyl-4-  isoxazo- 
lepropionic  acid  (AMPA),  histamine,  acetycholine,  and  7-aminobutyric  acid  (GABA).  One 
attractive  aspect  of  this  focus  on  postsynaptic  receptors  is  it  facilitates  mathematical  anal¬ 
ysis  on  the  basis  of  the  effect  of  receptor  binding  on  the  postsynaptic  potential.  This  is  in 
marked  contrast  to  the  Meyer- Overton  hypothesis,  which  failed  to  explicitly  detail  how  a 
nonspecific  perturbation  of  the  lipid  membrane  would  result  in  the  anesthetic  state. 

In  parallel  with  the  investigation  of  the  molecular  interactions  of  general  anesthetic 
agents,  there  has  also  been  active  investigation  of  the  anatomic  pathways  involved  in  the 
transition  from  consciousness  to  anesthesia.  There  is  compelling  evidence  that  the  immo¬ 
bility  created  by  some  anesthetics  is  mediated  at  the  level  of  the  spinal  cord.  In  contrast, 
functional  imaging  and  electroencephalograph  analysis  has  suggested  that  the  site  of  sup¬ 
pression  of  consciousness  is  the  thalamus,  and  thalamocortical  tracts  may  play  a  critical  role 
in  the  suppression  of  consciousness. 

Despite  these  advances  in  our  understanding  of  the  molecular  interactions  of  anesthetic 
agents  and  of  specific  anatomic  loci  for  the  action  of  anesthetic  agents,  there  has  been 
less  development  of  a  mathematical  framework  to  understand  this  fascinating  and  clinically 
important  phenomenon.  It  is  certainly  possible  that  if  the  mechanism  of  general  anesthesia 
is  the  binding  of  the  anesthetic  agent  to  a  specific  receptor  protein,  then  the  nearly  all- 
or-none  transition  from  the  awake  state  to  the  anesthetized  state  could  be  explained  by  a 
highly  cooperative  binding  of  the  anesthetic  to  the  receptor.  In  fact,  it  has  been  common 
to  mathematically  model  the  probability  of  responsiveness  to  drug  concentration  using  the 
Hill  equation,  a  simplified  equation  originally  derived  in  1909  to  describe  the  cooperative 
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binding  of  oxygen  to  the  hemoglobin  molecule.  However,  to  date,  no  single  unifying  receptor 
mediating  general  anesthesia  has  been  identified. 

Rather,  the  most  likely  explanation  for  the  mechanisms  of  action  of  anesthetics  lies  in 
the  network  properties  of  the  brain.  It  is  well  established  that  there  are  two  general  types 
of  neurons  in  the  central  nervous  system — excitatory  and  inhibitory — interconnected  in  a 
complex  network.  The  action  potential  of  a  spiking  neuron  is  propagated  along  the  axon 
to  synapses  where  chemical  neurotransmitters  are  released  that  generate  a  postsynaptic 
potential  on  the  dendrites  of  connected  neurons.  Excitatory  neurons  generate  a  depolarizing 
postsynaptic  potential  on  the  dendrite  of  the  connected  neuron  and  if  the  depolarization  is 
of  sufficient  magnitude,  then  a  spike  will  be  induced  in  the  connected  neuron.  In  contrast, 
inhibitory  neurons  generate  a  hyperpolarizing  postsynaptic  potential;  an  effect  that  acts  to 
maintain  a  quiescent  state. 

The  human  central  nervous  system  involves  a  complex  large-scale  interconnected  neural 
network  involving  feedforward  and  feedback  (or  recurrent)  networks,  with  the  brain  serving 
as  the  central  element  of  this  network  system.  The  brain  is  interconnected  to  receptors  that 
transmit  sensory  information  to  the  brain,  and  in  turn  the  brain  delivers  action  commands 
to  effectors.  The  neural  network  of  the  brain  consists  of  approximately  1011  neurons  (nerve 
cells)  with  each  having  104  to  105  connections  interconnected  through  subnetworks  or  nuclei. 
The  nuclei  in  turn  consist  of  clusters  of  neurons  each  of  which  performs  a  specific  and  defined 
function. 

The  most  basic  characteristic  of  the  neurons  that  comprise  the  central  nervous  system  is 
the  electrochemical  potential  gradient  across  the  cell  membrane.  All  cells  of  the  human  body 
maintain  an  electrochemical  potential  gradient  between  the  inside  of  the  cell  and  the  sur¬ 
rounding  milieu.  Neurons  have  the  capacity  of  excitability.  If  stimulated  beyond  a  threshold 
the  neuron  will  “Ere”  and  produce  a  large  voltage  spike  (the  action  potential)  before  return¬ 
ing  to  the  resting  potential.  The  neurons  of  the  brain  are  connected  in  a  complex  network 
in  which  the  firing  of  one  neuron  can  be  the  stimulus  for  the  firing  of  another  neuron.  A 
major  focus  of  theoretical  neuroscience  has  been  describing  neuronal  behavior  in  terms  of 
this  electrochemical  potential,  both  at  the  single  neuron  level  but  more  ambitiously,  at  the 
level  of  multi-neuron  networks.  In  this  type  of  analysis  the  specific  properties  of  the  single 
neuron  that  are  most  relevant  are  how  the  spike  of  a  one  neuron  alters  the  electrochemical 
potential  of  another  neuron,  and  how  this  change  in  the  potential  results  in  a  neuronal  spike. 
The  physical  connection  between  neurons  occurs  in  the  synapse,  a  small  gap  between  the 
axon,  the  extension  of  the  cell  body  of  the  transmitting  neuron,  and  the  dendrite,  the  exten¬ 
sion  of  the  receiving  neuron.  The  signal  is  transmitted  by  the  release  of  a  neurotransmitter 
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from  the  axon  into  the  synapse.  This  neurotransmitter  diffuses  across  the  synapse,  binds  to 
a  postsynaptic  receptor  membrane  protein  on  the  dendrite,  and  alters  the  electrochemical 
potential  of  the  receiving  neuron. 

There  is  considerable  evidence  that  general  anesthetics  alter  postsynaptic  potentials.  For 
example,  it  is  possible  that  the  anesthetic  bifurcation  to  unconsciousness  or  the  nearly  all-or- 
none  characteristic  induction  of  anesthesia  is  a  type  of  phase  transition  of  the  neural  network. 
This  possibility  was  first  considered  by  Steyn-Ross  et  al.  Their  focus  was  on  the  mean  voltage 
of  the  soma,  or  cell  body,  of  neurons.  Specifically,  they  show  that  the  biological  change  of 
state  to  anesthetic  unconsciousness  is  analogous  to  a  thermodynamic  phase  change  involving 
a  liquid  to  solid  phase  transition.  For  certain  ranges  of  anesthetic  concentrations,  their  first- 
order  model  predicts  the  existence  of  multiple  steady  states  for  brain  activity  leading  to  a 
transition  from  normal  levels  of  cerebral  cortical  activity  to  a  quiescent,  low-firing  state. 

In  this  research  [16,39],  we  present  an  alternative  approach  to  the  possibility  of  neuronal 
network  phase  transition  in  terms  of  neuronal  firing  rates,  using  the  concept  of  multista¬ 
bility  for  dynamical  systems.  Multistability  is  the  property  whereby  the  solutions  of  a 
dynamical  system  can  alternate  between  two  or  more  mutually  exclusive  Lyapunov  stable 
and  convergent  states  under  asymptotically  slowly  changing  inputs  or  system  parameters. 
In  particular,  multistable  systems  give  rise  to  the  existence  of  multiple  (isolated  and/or  a 
continuum  of)  stable  equilibria  involving  a  quasistatic- like  behavior  between  these  multiple 
semistable  steady  states  [16].  Semistability  is  the  property  whereby  the  solutions  to  a  dynam¬ 
ical  system  converge  to  Lyapunov  stable  equilibrium  points  determined  by  the  system  initial 
conditions  [46].  Multistability  is  ubiquitous  in  biological  systems  ranging  from  biochemi¬ 
cal  networks  to  ecosystems  to  gene  regulation  and  cell  replication.  Since  molecular  studies 
suggest  that  one  possible  mechanism  of  action  of  anesthetics  is  the  inhibition  of  synaptic 
transmission  in  cortical  neurons,  this  suggests  that  general  anesthesia  is  a  phenomenon  in 
which  different  equilibria  can  be  attained  with  changing  anesthetic  agent  concentrations. 
Hence,  multistability  theory  can  potentially  provide  a  theoretical  foundation  for  describing 
general  anesthesia. 

In  this  research  [16,39],  we  present  a  thorough  discussion  on  biological  neural  networks. 
In  particular,  the  fundamental  building  block  of  the  central  nervous  system,  the  neuron, 
is  represented  as  a  dynamic  element  that  is  “excitable,”  and  can  generate  a  pulse  or  spike 
whenever  the  electrochemical  potential  across  the  cell  membrane  of  the  neuron  exceeds  a 
threshold  value.  More  specifically,  a  nonlinear  discontinuous  system  is  derived  for  describing 
the  relationship  between  the  synaptic  current  and  firing  rates  of  excitatory  and  inhibitory 
neural  networks.  Then,  we  develop  some  basic  results  on  differential  inclusions  which  pro- 
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vide  the  mathematical  foundation  for  discontinuous  dynamical  systems  generating  Filippov 
and  Caratheodory  solutions.  To  establish  convergence  and  semistability  for  discontinuous 
dynamical  systems  we  introduce  the  notion  of  nontangency  between  a  discontinuous  vector 
held  and  a  weakly  invariant  or  weakly  negatively  invariant  subset  of  the  level  or  sublevel  sets 
of  a  Lyapunov  function.  Specifically,  to  capture  the  notion  of  nontangency  we  introduce  the 
direction  cone  of  a  discontinuous  vector  held  [9]. 

Next,  we  use  positive  limit  sets,  restricted  prolongations,  and  nontangency  to  develop 
Lyapunov  analysis  for  convergence  and  semistability  to  establish  multistability  for  discon¬ 
tinuous  dynamical  systems.  Here,  the  restricted  prolongation  of  a  point  is  a  subset  of  its 
positive  prolongation.  In  particular,  we  establish  connectedness  and  invariance  properties 
of  restricted  prolongations,  and  give  inclusion  results  for  restricted  prolongations  in  terms 
of  invariant  and  negatively  invariant  subsets  of  the  level  sets  of  a  Lyapunov  function  and 
its  derivative.  Then,  using  nontangency,  we  obtain  Lyapunov  results  for  convergence  and 
semistability  to  develop  sufficient  conditions  for  multistability  for  discontinuous  dynamical 
systems. 

Finally,  we  apply  our  results  to  excitatory-inhibitory  bring  neural  models.  While  there 
is  ongoing  debate  as  to  whether  information  is  encoded  by  the  bring  rates  (i.e.,  rate-coding) 
of  spiking  neurons  or  by  precise  timing  of  single  neuron  spikes  (i.e.,  temporal  coding),  it 
is  evident  that  bring  rates  do  characterize  central  nervous  system  activity.  Firing  rates 
are  nonnegative  entities  and  the  nonnegativity  constraint  for  neural  network  activity  can  be 
easily  incorporated  within  nonlinear  dynamical  system  theories  using  solutions  of  differential 
equations  evolving  in  cones  [1].  Next,  we  develop  a  two-class  model  for  characterizing  mean 
excitatory  and  inhibitory  synaptic  drives  to  explain  the  underling  mechanism  of  action  for 
anesthesia  and  consciousness.  In  particular,  we  demonstrate  how  the  full  synaptic  drive 
bring  model  can  be  reduced  to  a  model  involving  mean  excitatory  and  inhibitory  synaptic 
drives,  and  then  use  our  results  to  show  multistability. 

3.  Research  Personnel  Supported 

Faculty 

Wassim  M.  Haddad,  Principal  Investigator 

Graduate  Students 

K.  Volyanskyy,  Ph.  D,  and  T.  Sadikhov,  Ph.  D. 

Several  other  students  (T.  Yucelen  and  A.  L’Afflitto)  were  involved  in  research  projects 
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that  were  closely  related  to  this  program.  Although  none  of  these  students  were  financially 
supported  by  this  program,  their  research  did  directly  contribute  to  the  overall  research 
effort.  Furthermore,  one  Ph.  D.  dissertation  was  completed  under  partial  support  of  this 
program;  namely 

K.  Volyanskyy,  Adaptive  and  Neuroadaptive  Control  for  Nonnegative  and  Compart- 
mental  Dynamical  Systems,  August  2010. 

Dr.  Volyanskyy  is  presently  a  postdoctoral  fellow  of  the  School  of  Civil  Engineering  at 
Georgia  Tech. 

4.  Interactions  and  Transitions 

4.1.  Participation  and  Presentations 

The  following  conferences  were  attended  over  the  past  three  years. 

American  Control  Conference,  St.  Louis,  MO,  June  2009. 

IEEE  Conference  on  Decision  and  Control,  Shanghai,  P.  R.  China,  December  2009. 
American  Control  Conference,  Baltimore,  MD,  July  2010. 

IEEE  Conference  on  Decision  and  Control,  Atlanta,  GA,  December  2010. 

American  Control  Conference,  San  Francisco,  CA,  June  2011. 

IEEE  Conference  on  Decision  and  Control,  Orlando,  FL,  December  2011. 

Furthermore,  conference  articles  [25-45]  were  presented. 

4.2.  Transitions 

Our  work  on  adaptive  and  neuroadaptive  control  of  drug  delivery  [1]  partially  supported 
under  this  program  continues  to  transition  to  clinical  studies  at  the  Northeast  Georgia  Medi¬ 
cal  Center  in  Gainsville,  Georgia,  under  the  direction  of  Dr.  James  M.  Bailey  (770-534-1312), 
director  of  cardiac  anesthesia  and  consultant  in  critical  care  medicine.  This  work  has  recently 
transitioned  from  operating  room  (OR)  hypnosis  to  intensive  care  unit  (ICU)  sedation.  In 
addition,  this  work  was  communicated  to  Colonel  Leopoldo  C.  Cancio  (210-916-3301)  of  the 
US  Army  Institute  of  Surgical  Research  in  Fort  Sam  Houston,  San  Antonio,  in  order  to  pro¬ 
vide  improvements  for  combat  casualty  care  in  current  and  future  battlefields.  Collaboration 
with  the  US  Army  Institute  of  Surgical  Research  is  underway. 
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